Feature Engineering Bookcamp by Sinan Ozdemir;
Author:Sinan Ozdemir; [Ozdemir, Sinan]
Language: eng
Format: epub
Publisher: Simon & Schuster
Published: 2022-08-24T22:00:00+00:00
Stemmed
Removed of any stop words
And in this case our resulting list of tokens is
['wait', 'plane']
We can now use this custom tokenizer by setting our TfidfVectorizerâs tokenizer parameter, as seen in listing 5.16. Note that because our tokenizer will lowercase and remove stop words for us, we wonât need to grid search for these parameters.
Listing 5.16 Using our custom tokenizer
ml_pipeline = Pipeline([ ('vectorizer', TfidfVectorizer(tokenizer=stem_tokenizer)), â¶ ('classifier', clf) ]) params = { # 'vectorizer__lowercase': [True, False], # 'vectorizer__stop_words': [], â· 'vectorizer__max_features': [100, 1000, 5000], 'vectorizer__ngram_range': [(1, 1), (1, 3)], 'classifier__C': [1e-1, 1e0, 1e1] } print("Stemming + Log Reg
=====================") advanced_grid_search( # remove cleaning train['text'], train['sentiment'], test['text'], test['sentiment'], ml_pipeline, params )
â¶ Using a custom tokenizer
â· Not needed anymore, as our tokenizer is removing stop words and is lowercasing
Our results (figure 5.19) show a reduction in performance, like we saw with our text cleaning.
Download
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.
Computer Vision & Pattern Recognition | Expert Systems |
Intelligence & Semantics | Machine Theory |
Natural Language Processing | Neural Networks |
Algorithms of the Intelligent Web by Haralambos Marmanis;Dmitry Babenko(8332)
Test-Driven Development with Java by Alan Mellor(7024)
Data Augmentation with Python by Duc Haba(6946)
Principles of Data Fabric by Sonia Mezzetta(6680)
Learn Blender Simulations the Right Way by Stephen Pearson(6592)
Microservices with Spring Boot 3 and Spring Cloud by Magnus Larsson(6450)
Hadoop in Practice by Alex Holmes(5977)
RPA Solution Architect's Handbook by Sachin Sahgal(5843)
Jquery UI in Action : Master the concepts Of Jquery UI: A Step By Step Approach by ANMOL GOYAL(5831)
The Infinite Retina by Robert Scoble Irena Cronin(5543)
Big Data Analysis with Python by Ivan Marin(5508)
Life 3.0: Being Human in the Age of Artificial Intelligence by Tegmark Max(5182)
Pretrain Vision and Large Language Models in Python by Emily Webber(4471)
Infrastructure as Code for Beginners by Russ McKendrick(4260)
Functional Programming in JavaScript by Mantyla Dan(4059)
The Age of Surveillance Capitalism by Shoshana Zuboff(3981)
WordPress Plugin Development Cookbook by Yannick Lefebvre(3966)
Embracing Microservices Design by Ovais Mehboob Ahmed Khan Nabil Siddiqui and Timothy Oleson(3769)
Applied Machine Learning for Healthcare and Life Sciences Using AWS by Ujjwal Ratan(3743)
